AD-A077  0»7 


UNCLASSIFIED  NRC-Tfft-ltM 


WISCONSIN  UNIV-MAOISON  MATHEMATICS  RESEARCH  CENTER  R/f 

QUASI-NEWTON  METHODS  FOR  6ENERALIZE0  EQUATIONS. <U> 

JUN  7f  M  H  JOSCFHY  OAAQ29-75-C-OOt* 


F/f  lt/1 


mil.  FILE  COPY. 


Mathematics  Research  Center 
University  of  Wisconsin-Madison 
610  Walnut  Street 
Madison,  Wisconsin  53706 


June  1979 


Received  May  25,  1979 


Approved  for  public  release 
Distribution  unlimited 


Sponsored  by 

U.  S.  Army  Research  Office 
p.  0.  Box  12211 
Research  Triangle  Park 
North  Carolina  27709 


National  Science  Foundation 
Washington,  D.  C.  20550 


UNIVERSITY  OF  WISCONSIN  -  MADISON 
MATHEMATICS  RESEARCH  CENTER 

OUASI-NEWTON  METHODS  FOR  GENERALIZED  EOUATIONS 
Norman  H.  Josephy 


Sponsored  by  the  United  States  Army  under  Contract  No.  DAAG29-75-C-0024 . 
This  material  is  based  upon  work  supported  by  the  National  Science  Founda¬ 
tion  under  Grant  No.  DCR  74-20584  and  Grant  No.  MCS  74-20584  A02  and  the 
Graduate  School  of  the  University  of  Wisconsin  -  Madison. 


Significance  and  Explanation 


Many  practical  problems  in  operations  research  and  mathematical 
economics  can  be  formulated  as  a  system  of  equations  and  inequalities. 
A  previous  paper  (Josephy  [10])  developed  an  iterative  procedure, 
Newton's  method,  for  computing  a  solution  to  such  a  system.  However, 
situations  arise  in  which  it  is  computationally  unreasonable  to 
evaluate  the  derivative  appearing  in  the  Newton  iteration  formula. 

In  such  cases,  an  approximation  to  the  derivative  which  is  easier  to 
compute  is  used  in  place  of  the  derivative.  The  iterative  procedure 
with  an  approximation  to  the  derivative  is  called  a  quasi-Newton 
method.  This  paper  extends  the  results  known  about  the  convergence 
and  rate  of  convergence  of  quasi-Newton  methods  for  equations  to  the 
case  of  equations  and  inequalities  which  arise  in  the  nonlinear 
programming  problem  and  the  economic  equilibrium  problem.  The 
method  is  illustrated  by  solving  a  small  practical  problem  involving 


The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive 
summary  lies  with  MRC,  and  not  with  the  author  of  this  report. 


OUAS I -NEWTON  METHODS  FOR  GENERALIZED  EQUATIONS 


Norman  H.  Josephv 

1 .  Introduction . 

We  recall  the  definition  of  a  generalized  equation.  Further 
elaboration  can  be  found  in  Robinson  [13-17]  and  Josephy  [10]. 


Let  C  be  a  non-empty,  convex,  closed  subset  of  Rn.  The  normal 
cone  to  C  at  x  t  C  is  the  set  of  outward  pointing  normals  to  C  at  x. 

We  have  the  following: 

Definition  1.  Let  C  be  a  non-empty,  closed,  convex  subset  of  Rn  . 


The  normal  cone  to  C  at  x  is  given  by 
Nc(x):  =  f  {z|  <  z,k-x>  =  0V  k  e  C) 


if  x  e  C, 
if  x  $  C 


We  can  now  define  a  generalized  equation. 

Definition  2.  Let  f:D  c  Rn  ■+■  Rn. 

Let  C  be  a  non-empty,  closed,  convex  subset  of  Rn  . 


A 


zeneralized  equation 


is  a  set  relation 


(1) 


0  e  f(x)  +  Nc(x)  . 


Thus,  x*  satisfies  the  generalized  equation  0  e  f(x)  +  N^Cx)  if 
and  only  if  x*  satisfies  the  relations 

x*  c  C 


and 


<  f(x*)t  k  -  x*  >  >  0  for  all  k  c  C  . 
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Newton's  method  for  solving  a  generalized  equation  0  e  F(x)  +  N^,(x) 

involves  the  evaluation  of  F'  ,  the  derivative  of  F  ,  at  each  iterate 

x  .  (See  Josephy  [10]).  In  situations  where  this  evaluation  imposes  an 
n 

excessive  computational  burden,  we  consider  replacing  F'Cx^)  with  an 
approximation.  A  quasi-Newton  method  for  solving  0  c  F(x)  +  N^(x)  re¬ 
places  F  with  an  affine  map  F(x^)  +  BO-x^)  whose  derivative  B  is 
both  an  approximation  to  F'(xn)  in  some  sense  and  is  computationally 
easier  to  evaluate.  One  class  of  approximations,  the  secant  approxima¬ 
tions,  has  been  the  subject  of  great  interest  to  those  solving  systems  of 
equations  and  related  problems,  such  as  non-linear  least  squares  estima¬ 
tion,  unconstrained  and  equality-constrained  optimization,  and,  most 
recently,  inequality-constrained  optimization.  A  sample  of  the  recent 
work  on  quasi-Newton  methods  can  be  found  in  Brodlie  [2],  Dennis  and  Mord 
[5],  Gill  and  Murray  [8],  Tapia  [18],  and  Han  [9]. 

In  this  paper,  we  will  extend  to  the  generalized  equation  case  two 
fundamental  results  on  the  convergence  and  rate  of  convergence  of  quasi- 
Newton  methods  with  secant  approximations.  Section  2  contains  intro¬ 
ductory  material  on  quasi-Newton  methods.  We  prove  local  convergence  of 
quasi-Newton  iterates  to  a  solution  of  a  generalized  equation  in  Section  3, 
and  prove  in  Section  4  Q-superlinear  rate  of  convergence  for  the  quasi- 
Newton  iterates.  We  conclude  in  Section  5  with  the  computational  results 
of  applying  a  quasi-Newton  method  to  the  generalized  equation  representing 
a  nonlinear  complementarity  problem  of  Kojima  [11]. 
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i 


lasi-Newton  Methods. 


Newton's  method  for  finding  a  zero  of  F  proceeds  by  itera¬ 
tively  solving  0  =  F (x  )  +  F'fx  )  (x-x  )  for  a  solution  x  ..  A 
7  °  n  ir  v  n  n+1 

quasi-Newton  method  iteratively  solves  0  =  Ffx^)  *  ®n^x'xn^»  w^ere 
approximates  F'(xn)  in  some  sense.  An  approximation  of  much  cur¬ 
rent  interest  is  a  generalization  to  maps  on  Rn  of  the  secant  approxi¬ 
mation  of  the  derivative  of  a  real-valued  map  on  R.  Let  f:R  -*■  R  have 
derivative  f ' .  Given  two  points  in  R,  say  x^  and  x^^,  the  secant 
approximation  to  f' (xn+j)  is 

b;  ■  -  f<x„»/(Vl  -  xn>  ' 

Alternatively,  the  secant  approximation  b  is  the  unique  solution  of 


f(xn.l’  -  f(xn>  ■  b(xn.l  *  x„> 


.n  .  „n 


For  F:R  -*■  R  ,  a  secant  approximation  to  F'(x  j)  is  any  matrix  B 


satisfying 


P<Vl>  •  P<V  =  B(Vl  -  Xn^  • 


Since  equation  (2)  does  not  uniquely  specify  B,  additional  conditions 
can  be  imposed  to  guarantee  desired  properties  of  B,  such  as  symmetry 
and  positive  definiteness.  This  approach  to  secant  approximations 
is  discussed  in  more  detail  in  Dennis  and  Schnabel  [61  .  Stable 

numerical  techniques  for  efficiently  implementing  these  secant  ap¬ 
proximations  are  described  in  Gill,  Golub,  Murray  and  Saunders  [71. 
The  price  one  pays  for  using  an  approximation  for  F'(xn)  is  the 


/  -i" 


loss  of  the  quadratic  rate  of  convergence  to  a  local  solution.  How¬ 
ever,  iterates  generated  by  secant  approximations  typically  exhibit 
Q-superl inear  convergence,  provided  appropriate  conditions  are  satis¬ 
fied.  We  will  need  the  concepts  of  linear  and  Q-superlinear  con¬ 
vergence,  which  we  now  define. 


Definition  1.  A  sequence  {x^}  converges  to  x*  linearly  (Q-super- 

linearly)  if  and  only  if  for  some  a  e  (0,1)  (for  some 
sequence  {a^}  converging  to  zero), 

||  xn+1  -  x*  ||=  a||  xn-x* || 


r\  m 

n+1 


The  Q-superlinear  convergence  of  quasi-Newton  iterates  can  be  proven 
in  the  following  fashion.  In  Theorem  1,  we  establish  linear  convergence 
for  iterates  determined  by  any  quasi-Newton  method  whose  approximations 
{B^}  satisfy  a  certain  norm  inequality.  The  proof  for  the  generalized 
equation  case  is  modeled  on  the  single-valued  case  given  in  Broyden, 

Dennis  and  Mor£  [1,  Theorem  3.2].  We  can  then  invoke  the  results 
appearing  in  Broyden,  Dennis  and  Mor€  [1]  which  show  that  the  traditional 
update  formulas,  such  as  the  Broyden  rank  one,  Powell  symmetric  Broyden 
rank  two,  and  the  DFP  update  of  {B^}  (see  Dennis  and  Mor4  [5]  for  further 
details)  satisfy  the  required  norm  inequality.  This  establishes  a  linear 
convergence  rate  for  the  generalized  equation  case.  Dennis  and  Mord  [4, 
Theorem  3.4  and  Section  4]  show  that 

l  :=  lim  II  (B  -F’(x*))(x  ^.-x  >11  /II  x  ^,-x  II  =0  , 
n  n+1  n  n+1  n  ’ 
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whenever  the  sequence  {B^}  is  computed  by  one  of  the  above  mentioned 

update  formulas  and  the  sequence  {x^}  converges  linearly  to  x*  . 

They  note,  in  a  remark  immediately  following  their  proof  of  Theorem  3.4, 

that  the  limit  5=0  is  established  independent  of  how  the  sequence  {x^} 

is  generated.  In  particular,  their  result  remains  valid  when  {x  }  is 

n 

the  linearly  convergent  sequence  of  solutions  to  the  quasi-Newton  method 
applied  to  generalized  equations  rather  than  to  single-valued  equations. 
Thus,  for  the  update  formulas  mentioned  above,  the  limit  £=0  is  valid 
for  the  generalized  equation  case,  since  Theorem  1  of  this  paper  estab¬ 
lishes  the  linear  convergence  of  {x^}.  Theorem  2  shows  that  ?.=0  suf¬ 
fices  to  establish  Q-superlinear  convergence  of  iterates  {x  1  generated 

n 

by  quasi-Newton  methods  applied  to  generalized  equations.  We  note  that 
proofs  of  the  two  theorems  appearing  in  this  paper,  when  restricted  to 
the  single-valued  equation  case,  have  appeared  in  the  literature  (Dennis 
and  More  (51).  However,  those  proofs  depend  critically  upon  the  fact  that 
single-valued  equations  are  being  solved.  Hence,  proofs  are  given  in  this 
paper  which  are  valid  for  the  generalized  equation  case. 

We  conclude  this  section  with  a  corollary  to  Theorem  2.4  of  Robinson 
[16],  and  a  lemma  from  Ortega  and  Rheinboldt  [12]. 

Corollary  1.  Let  C  be  a  closed,  convex,  nonempty  subset  of  Rn  ,  and 
let  D  be  an  open,  convex,  nonempty  subset  of  Rn  .  Let  f  :D  -*•  Rn  have 
Frdchet  derivative  f.  Suppose  the  generalized  equation  0  e  f(x)  +  N^,(x) 
has  a  strongly  regular  solution  at  x*  e  D  ,  with  associated  Lipschitz 
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constant  d  . 


r-o  1 1— • 


Then,  for  some  positive  constants  r,  R,  b  and  e  ,  the  following  hold. 

Let  A  be  an  n*n  matrix  and  let  x  e  Rn.  Then 
B(x*,r)  n  (f  (x)  +  A*(*-x)  +  N^.)  *  restricted  to  B(0,R) 
is  single-valued  and  Lipschitz  continuous  with  modulus 
dfl-d  ||A-f '  (x*)  ||  )  *  ,  whenever ||  x-x* || <  e  and 
||A-f,(x*)||  <  b  . 

Lemma  1 .  Let  D  be  an  open,  convex  subset  of  Rn  .  Let  f:D  ■>  Rn  be 

continuously  differentiable.  Suppose  that  for  some  k  >  0  , 

llf'(u)  -  f'(v)ll  <  kJI  u  —  v  II  whenever  u,v  e  D  .  Then  llf(u)  -  Lf  ^  (u)  II  < 
2 

kllu  -  vll  whenever  u,v  e  D  ,  where  Lfv(u)  :=  f(v)  +  f'(v)(u-v). 

The  definition  and  properties  of  strong  regularity  of  a  solution  to  a 
generalized  equation  can  be  found  in  Robinson  [16].  The  use  of  strong 
regularity  in  establishing  convergence  properties  of  Newton's  method  for 
generalized  equations  can  be  found  in  Josephy  [10]. 


I 

I 

I 

I 

I 


! 
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3 .  Local  Convergence. 

We  begin  with  a  definition. 

Definition  2.  (Dennis  and  liord  [5]). 

Let  D  be  an  open,  nonempty  convex  subset  of  Rn,  and 

let  P  denote  a  class  of  n  x  n  matrices.  Then  an 

update  function  U  is  a  map  from  D  x  p  to  subsets  of  P. 

An  update  function  can  be  used  to  define  a  set  of  approximations  to 

the  derivative  of  F  at  xn+-^»  In  the  typical  situation,  the  present 

iterate  xn  and  the  present  approximation  Bn  will  determine  the  new 

iterate  x  . .  Both  iterates  x  and  x  ,  are  used  to  determine  the 
n+1  n  n+1 

next  approximation  B  This  procedure,  by  which  Bn+J  is  determined 

from  xfi  and  B^,  can  be  represented  by  B^+1  e  UCx^.B^),  where  U  is 
the  appropriately  defined  update  function. 

We  now  state  and  prove  the  local  convergence  theorem. 

Theorem  1 .  Let  D  be  an  open,  nonempty  convex  subset  of  Rn,  and  let 
C  be  a  closed,  nonempty  convex  subset  of  Rn.  Let  F:D  Rn  have  a 
Lipschitz  continuous  derivative  F'  with  Lipschitz  constant  K.  Sup¬ 
pose  x*  e  D  is  a  strongly  regular  solution  of  the  generalized  equa¬ 
tion  0  e  F(x)  +  N^(x)  with  associate  Lipschitz  constant  d.  Let  W 
be  an  open  neighborhood  of  F'(x*)  in  the  space  of  linear  maps  from 
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Rn  to  Rn .  Let  ||  •  ||M  denote  a  matrix  norm  and  let  a  >  0  be  such  that 
11*11  =  all'llM>  where  ||  •  ||  is  a  matrix  norm  subordinate  to  the  given 
vector  norm  on  Rn.  Suppose  that  two  positive  constants,  a^  and  a^, 
exist  such  that  the  inequality 

II  B  -  F*(x*)||  M  =  (1+aj  *max{  ||  x-x*||,||  x-x* ||  >  )  ||  B-F'(x*)ll  M 

+  a2*max{  ||  x-x*  ||  ,  ||  x-x*  || } 

holds  whenever  the  following  conditions  are  valid: 

(x,B)  e  D  x  w  ,  B  e  U(x,B),  where  U  is  an  update  function, 
and  x,  the  vector  closest  to  x  in  the  set 
(F (x)  +  B(*-x)  +  Nc)  *(0),  exists. 

Let  b,  e,  r  and  R  be  the  positive  constants  associated  with  the 
strongly  regular  solution  x*,  as  given  in  Corollary  1  .  Fix  p 
positive  and  less  than  1.  Suppose  the  following  relations  hold,  with 
e  and  b  reduced,  if  necessary,  from  those  guaranteed  by  Corollary  1  . 

II  Bo~F*  Cx^||  M  <  iba'1  and  ||  xQ-x*||  <  e. 

(■^Ke+b)e  <  R  and  d(l-db)  *  (-^Ke+b)  <  p. 

2a(a  ^a^+a^e  (1-p)  1  <  b  and  B(x*,e)  c  D. 

Then  a  sequence  of  iterates  {x  }  and  a  sequence  of  matrices  {B  } 

n  n 

exist  and  satisfy  the  following  relations,  for  all  n  =  0  . 
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0  e  F(x  ) 
n 


+  B  (x  -x  )  +  N  (x  ) 
n  n  +  1  n  C  n' 


II  Bn-F'(x*)||  M  <  ba¬ 

ll  vrx*H  “  pU  vx*  II 


Proof.  We  will  use  the  following  notation. 


Sn:  =  B(x*,r)  n  (F(x*)  +  Bn(-x*)  +  Nc) 


-1 


T  :  =  B(x*,r)  n  (F(xJ  +  B  f -xj  ♦  Nr) 


-1 


n  n  n 


J  :  =  F  (x*l  -  F(x  )  -  B  (x*  -  x  )  . 

n  '  v  n'  n  n' 


d:  =  d(l-db) 


-1 


By  hypothesis ,  ||  Bq-F'(x*)||  <  \ba  *  and  ||  XQ-x*||<e  .  Thus, 

||  Bq-F  1  (x*)  ||  <  b  .  By  Corollary  1  ,  TQ  restricted  to  B(0,R)  is 
single-valued  and  Lipschitz  continuous  with  Lipschitz  constant 
d ( 1  -d ||  Bq-F'(x*)||  )  *,  which  is  less  than  d.  Similarly,  re¬ 
stricted  to  B(0,R)  is  single-valued  and  Lipschitz  continuous  with 
Lipschitz  constant  d,  and  Sq(0)  =  x*.  Define  x^:  =  Tg(0).  In  order 
to  estimate  the  distance  from  x^  to  x*,  we  need  to  estimate  the  norm 
of  Jq.  It  follows  from 

V  -  -  F<V  -  VX*"V 

=  F (x*)  -  LF  (x*)  ♦  (F'(x*)-B0)(x*-x0)  4 
♦  F'(x0)  -  F'(x*)Cx*-xq) 
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that  the  norm  of  is  bounded  by 

II  J0H  =  ||f (x*)  -  lfXqCx*)  II  +  ||f '  (x*)  -  BQ ||  •  ||  x*-x0||  + 

♦  IlF'Cx*)  -  F'(x0)||.||x*-x0|| 

*  k||  X*-XQ  11 2  +  b II  x*-x0||  +  K||  x*-xQ||2 

=  (|  Ke+b)  ||  x*-xQ||  <  R  . 

By  definition  of  T^,  x^  e  B(x*jr)  and 

0  E  +  WV  +  NC^X1^ 

=  F(x*)  +  BqCXj-x*)  -  J0+Nc(x1) 

Thus,  JQ  e  F (x*)  +  Bgfx^x*)  +  N^x^  ,  so  that  x1  e  S0(JQ)-  But 

II  J0  II  <  R»  implying  S0(JQ)  is  a  singleton  and  =  SQ(J0).  We  can 

now  estimate  the  distance  between  x^  and  x*  as 

II  vx*H  =  IIVV  -  so(0)ll  “  aU  Jo>l 

=  d(|  Ke+b)  ||  x*-xQ||  <  p II  xQ-x*||  . 

We  now  proceed  by  induction.  We  will  show  that  for  all  k  =  0  , 

(3)  II  Bk  -  F’(x*)||  M  <  ba-1  , 

(4)  II  xic+rx*H  ■  pH  vx*H  *  pkH  vx*H  • 

We  have  already  established  (3)  and  (4)  when  k=0.  We  thus  assume 
that  (3)  and  (4)  hold  for  all  k  =  m-1,  where  m=l,  and  will  establish 
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(3)  and  (4)  for  k=m.  We  first  note  that  p  <  1  and  (4)  imply 


mk:  =  max{  ||  xR+1-x*||  ,  ||  xR-x*  || }  =  ||xk-x*|| 
for  all  k  =  m-1,  and 

II  Bk+rF’(X*)H  M  'll  VF’(X*)II  M  =  aimkH  Bk-F’Cx*)||  M  +  aA 

=  a1ba_1pk||  xQ-x*||  +  a2pk||  xQ-x*|| 

<  ,  ,  -1  .  k 

=  (a^a  +  a2)p  e  . 

Summing  over  k  between  1  and  m-1  yields 

II  VF,(X*)II  M  =  llB0-F^x*)ll  M  +  Calba_1  +  a2)(l-P)'le  • 

By  hypothesis,  ]|Bq-F  '  (x*)  ||  M  <  Iba-1  and  (a^ba  *+a2) (1-p)  *e  <  jba"1, 

hence  ||  Bm-F'(x*)||  ^  <  ba  *  .  Thus  (3)  is  established.  To  prove 
(4),  we  note  that  [|  x  -x*||  =  pm||  xQ-x*||  =  pme  <  e.  We  can  now  apply 

Corollary  1  to  T  and  S  and  conclude  that,  when  restricted  to 
m  m 

B(0,R),  both  are  single-valued  and  Lipschitz  continuous  with  Lipschitz 
constant  d(l-d||  B  -F'(x*)||)  Let  x  ,  :  =  T  (0)  and  note  that 

x*  =  S  (0).  It  remains  to  bound  the  distance  between  x  ,  and  x*. 
mv  J  m+1 

We  will  first  obtain  a  bound  on  the  norm  of  J  ,  and  then  use  that 

m 

bound  to  estimate  llxm+i"x*ll  •  We  ^ave  t*iat 
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J  :  =  F (x*)  -  F(x  )  -  B  (x*-x  ) 
m  ’  K  m  ml  m 


=  (F (x*)  -  LFx  (x*))  +  (F’(x*)  -  Bm)(x*-xm)  + 
m 


♦  (F'(xm)  -  F»(x*)(x*-xm)  . 


Using  Lemma  1  to  bound  the  first  term,  (3)  to  bound  the  second  term, 
and  Lipschitz  continuity  of  F'  to  bound  the  third  term,  we  have 

II  JJ  =  *KII  XnTx*l|2  +  b||  X  -x*||  +  K ||  x*-x  || 2 


=  (|Ke+b)  ||  xm-x*  ||  <  R  . 


By  definition  of  T  ,  x  ,  e  B(x*,r)  and 
/  m  m+1  v  ’  J 


0  e  F(x  )  +  B  (x  ,-x  )  +  N„(x  ,) 
m'  m v  m+1  nr  C  m+1' 


=  F(x*)  +  B  (x  ,-x*)  -  J  +  N_(x  . 

mv  m+1  ‘  m  m+1' 

Hence,  x .  e  S  fJ  )  and  ||j  ||  <  R,  from  which  we  can  conclude  that 
m+i  mm  "  m" 

xm+j  =  We  finish  the  induction  proof  by  noting  that 


x  .  -x*  ||  =  ||S  (J  )  -  S  (0) 
m+1  11  "mm  m 


=  d(l-d||  Bm  -  F '  (x*)  ||  y'w  Jm| 


=  d(|Ke+b)  ||  xm-x*| 


<  pH  vx*II  • 


This  completes  the  induction  and  the  proof  of  the  theorem. 


/  .  >;  '■ 


4 .  Q-Superlinear  Convergence 


The  preceding  theorem  establishes  local  linear  convergence 
of  the  quasi-Newton  iterates  {x^}  to  a  strongly  regular  solution  x* 
of  the  generalized  equation,  for  any  update  function  which  satisfies 
the  given  norm  inequality.  The  results  of  Broyden,  Dennis  and  More  [1] 
show  that  the  traditional  update  functions  satisfy  this  inequality. 
Dennis  and  More  [4]  show  that  for  the  traditional  update  functions, 
the  limit  appearing  in  the  next  theorem  is  zero.  Theorem  2 
proves  that  this  suffices  for  the  quasi-Newton  iterates  to  converge 
Q- super linearly. 

Theorem  2.  Let  D  be  an  open,  nonempty  convex  subset  of  Rn,  and  let 
C  be  a  closed,  nonempty  convex  subset  of  Rn.  Let  F  have  a  Lipschitz 
continuous  derivative  F'  with  Lipschitz  constant  K.  Suppose  that 
the  generalized  equation  0  e  F  (x)  +  N^(x)  has  a  strongly  regular  solu¬ 
tion  x*  with  associated  Lipschitz  constant  d.  Let  {B^}  be  a 
sequence  of  n  x  n  matrices.  Assume  that  the  set  of  norms  {j|Bk||} 
is  bounded.  Let  xQ  e  D  and  suppose  that  the  sequence  {x^}  ,  k  =  0 
satisfies  the  relation  0  c  F(xk)  +  Bk(xk+1-xk)  +  Nc(xk+1). 

Also  assume  that  {xk}  converges  to  x*.  Define 

V  =  VF,(X‘>  ’  V  =  Vrxk  '  and  Vr  =  -F(V  -  Vk  • 

I!  Ekskll 

Then  lim  -w-— — jp  =  0  implies  that  the  sequence  {xk}  converges 

k-*»  k 
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*' r' 


Q-superlinearly  to  x*. 


Proof.  By  definition  and  some  algebra,  we  have 


Eksk =  F(JW  -  F(V  -  F'(x*)sk  -  (vk+i  *  F°W} 


Letting  P|c+j:  =  +  we  can  so^ve  f°r  P^+i  anc*  ta^e  norms 


to  obtain 


II  Pk+1ll  <  II A  F  ||  ||  Eksk| 


where  AF:  =  -  F(xk)  -  F'  (x*)  (xk+j-xk)  .  But  the  Lipschitz 

continuity  of  F'  implies  (see  Ortega  and  Rheinboldt  [12]  ) 

||AF||  =  K*max{  ||  xR+1-x*||  ,  ||  xk-x*  ||  }  •  ||  xk+1-xj|  .  Hence, 


=  lim  K-max  { ||  x. 


xk-x*||}  ♦ 


♦  lim 
k-*- 


To  obtain  the  next  result,  we  use  a  special  case  of  the  Implicit  Func¬ 
tion  Theorem  of  Robinson  (  16  ,  Theorem  2.1).  Specifically,  we  take 
as  the  function  f(p,x)  in  that  theorem  the  function  -p  ♦F(x)  .  The 
conclusions  of  that  theorem  give  us  the  following  results.  Fixing 

e  >  0  ,  there  exist  neighborhoods  U  of  0  and  W  of  x*  ,  and  a  single- 

e  e 

valued  map  x:  Ue  ■*  such  that  x(p)  is  the  unique  solution  in 


of  the  generalized  equation  0  e  -p  +  F(x)  +  N^Cx)  ■  Also,  for  any 
p,  q  e  Ue>  ||  x(p)  -  x (q)  ||  =  (d+e)  ||  p+q||  .  We  now  show  that  this 

result  can  be  applied  to  the  generalized  equation  0  e  F(xR)+BRsR  + 

N„fx.  ,)  ,  where  x.  ,  is  the  unknown  variable,  to  obtain  a  bound  on 
Cv  k+1  k+1 

||  xR+j  -  x*  ||  .  Note  that  the  assumptions  that  (|!  ||  }  is  bounded 

and  {xR}  converges  to  x*  imply  the  convergence  of  {vR+J}  to  -F(x*). 
Thus,  {pR+j}  converges  to  zero,  and  will  be  in  Ue  for  all  sufficiently 
large  k.  Also,  x(0)  =  x*.  Thus,  xR+1  e  We  , 

0  c  F(V  +  Bksk  ♦  Wi5  =  F(Vi)  -  Pk+1  *  Nc(3W  and 

j>k+l  e  Ug  for  all  sufficiently  large  k,  which  implies  x(pR+1)=  xR+J 
for  all  sufficiently  large  k.  Hence, 


II  Vill 
II  sk  II 


(d+e) 
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5.  Computational  Results. 


B  + 
n 


B 


n+l 


(v  -  B  S  )sn 

n  n  n  ii 


S 

n 


) 


where  S 

n 


x 

n 


and  v  •»f(x11)-f(x). 
n  n+l  n 

We  note  that  if  S  =0,  then  x  =  x  .  Since,  bv  definition, 
n  n+1  n 

0  c  f(xn)  +  VXn+l  "  V  +  N  4(Vl)*  replacln*  *n+l  by  xn 

R+ 

yields  0  e  f(x  )  +  M  . (x  ),  so  that  x  is  a  solution  to  the  original 
J  n  ^4  n  n 

nonlinear  complementarity  problem  if  =  0,  and  the  iterative  procedure 

is  terminated. 

The  results  of  applying  Lemke's  algorithm  to  the  linear  complemen¬ 
tarity  problems  with  Broyden  rank  one  approximations  to  the  derivative 
f'  are  given  in  Table  1. 


Table  1  Koj ima  Example 


Initial  x 

1. 

3. 

5. 


Iterations 

Final  x 

Final  f 

6 

(1.2247449,0,0, 

.49999993) 

(.2.10-6,  3.224745, 

5.0000002, 

-.9.10 

11 

(1.2247449,0,0, 

.49999995) 

(  0,  3.224745, 

5.0000000, 

-,9.]o 

12 

(1.2247449,0,0, 

.49999999) 

(-. 2 . 10*^ ,3.224745, 

4.9999998, 

-.9.10 

Three  starting  points,  with  each  component  of  x  initialized  to  the  entry  in 
column  1,  all  lead  to  a  convergent  sequence  of  iterates.  The  number  of  iterations 
is  listed  in  column  2,  with  the  final  value  of  x  and  f(x)  given  in  columns  3 
and  4,  respectively.  Each  iteration  consists  of  Lemke's  algorithm  applied  to  a 
4x4  linear  complementarity  problem,  each  of  which  required  two  pivot  operations 
of  Lemke's  algorithm  to  solve. 
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